Add IDocumentExtractionClient document-extraction capability as a new Microsoft.Extensions.DocumentExtraction library - #7588
Open
luisquintanilla wants to merge 16 commits into
Conversation
…ions.AI Introduces IOcrClient as a provider-neutral OCR/document-parsing capability in Microsoft.Extensions.AI.Abstractions, following the same abstraction + builder + middleware + DI shape as the existing capability family (IChatClient, ISpeechToTextClient, etc.). Abstractions (Microsoft.Extensions.AI.Abstractions): - IOcrClient, DelegatingOcrClient, OcrClientExtensions, OcrClientMetadata - OcrOptions, OcrResult, OcrPage, OcrTable, OcrTableCell, OcrBlock, OcrBoundingRegion, OcrUsage, OcrProgress Middleware + DI (Microsoft.Extensions.AI): - OcrClientBuilder, AsBuilder, AddOcrClient/AddKeyedOcrClient - LoggingOcrClient, OpenTelemetryOcrClient, ConfigureOptionsOcrClient and their builder extensions, mirroring the ISpeechToTextClient template All public surface is marked [Experimental] under the MEAI001 (AIOcr) diagnostic id. Includes unit tests for both libraries and updated API baselines.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new provider-neutral OCR / document-extraction capability to the Microsoft.Extensions.AI capability family, including abstractions, builder-based middleware pipeline, DI registration helpers, and OpenTelemetry/logging integrations.
Changes:
- Introduces
IOcrClient+ OCR result/options model types (pages, tables, blocks, bounding regions, usage/progress) inMicrosoft.Extensions.AI.Abstractions. - Adds
OcrClientBuilder+ DI registration extensions, plus middleware clients for logging, OpenTelemetry, and default-options configuration inMicrosoft.Extensions.AI. - Adds unit tests and updates API baseline JSONs and shared diagnostic IDs / OpenTelemetry constants to cover the new capability.
Reviewed changes
Copilot reviewed 40 out of 40 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/OpenTelemetryOcrClientTests.cs | Verifies OCR OpenTelemetry spans/tags and ActivitySource service exposure. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/OcrClientDependencyInjectionPatterns.cs | Validates DI registration and middleware wrapping patterns for OCR clients. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/OcrClientBuilderTests.cs | Tests builder pipeline ordering, null-handling, and service-provider flow-through. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/LoggingOcrClientTests.cs | Tests logging middleware behavior across log levels and DI-resolved logger factory. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Ocr/ConfigureOptionsOcrClientTests.cs | Tests options-cloning/configuration middleware behavior. |
| test/Libraries/Microsoft.Extensions.AI.Tests/Microsoft.Extensions.AI.Tests.csproj | Shares TestOcrClient into the non-abstractions test project. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/TestOcrClient.cs | Adds an OCR test double for abstraction-level tests. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrResultTests.cs | Tests OcrResult construction and markdown aggregation behavior. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrOptionsTests.cs | Tests cloning semantics for OcrOptions. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrClientMetadataTests.cs | Tests metadata property round-tripping and nullability behavior. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrClientExtensionsTests.cs | Tests DataContent overload and argument validation in extensions. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/OcrBoundingRegionTests.cs | Tests bounding-region helpers and bounds calculation. |
| test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/Ocr/DelegatingOcrClientTests.cs | Tests delegating client pass-through semantics and GetService behavior. |
| src/Shared/DiagnosticIds/DiagnosticIds.cs | Adds a new experimental diagnostic ID bucket for OCR (AIOcr). |
| src/Libraries/Microsoft.Extensions.AI/OpenTelemetryConsts.cs | Adds a non-standard GenAI usage attribute for pages processed. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OpenTelemetryOcrClientBuilderExtensions.cs | Adds .UseOpenTelemetry(...) builder extension for OCR pipelines. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OpenTelemetryOcrClient.cs | Implements OpenTelemetry instrumentation for OCR operations (spans + metrics). |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OcrClientBuilderServiceCollectionExtensions.cs | Adds AddOcrClient / AddKeyedOcrClient DI registration helpers. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OcrClientBuilderOcrClientExtensions.cs | Adds IOcrClient.AsBuilder() convenience extension. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/OcrClientBuilder.cs | Adds the OCR pipeline builder implementation (Use(...), Build(...)). |
| src/Libraries/Microsoft.Extensions.AI/Ocr/LoggingOcrClientBuilderExtensions.cs | Adds .UseLogging(...) builder extension for OCR pipelines. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/LoggingOcrClient.cs | Adds logging middleware for OCR operations with sensitive-data gating. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/ConfigureOptionsOcrClientBuilderExtensions.cs | Adds .ConfigureOptions(...) builder extension for defaulting/cloning options. |
| src/Libraries/Microsoft.Extensions.AI/Ocr/ConfigureOptionsOcrClient.cs | Implements options-configuration middleware for OCR operations. |
| src/Libraries/Microsoft.Extensions.AI/Microsoft.Extensions.AI.json | Updates API baseline to include OCR middleware/builder/DI additions. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Utilities/AIJsonUtilities.Defaults.cs | Registers OCR types for source-generated JSON serialization defaults. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrUsage.cs | Adds OCR usage model (pages processed + additional properties). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrTableCell.cs | Adds structured table-cell model (indices, spans, content, kind). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrTable.cs | Adds structured table model (cells or markdown + optional geometry). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrResult.cs | Adds OCR result model (pages + markdown aggregation + usage/raw/additional). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrProgress.cs | Adds progress model for long-running OCR operations. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrPage.cs | Adds per-page OCR model (markdown + tables/blocks/confidence). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrOptions.cs | Adds request options model (model id, include images, additional properties). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrClientMetadata.cs | Adds client metadata model (provider name/uri/default model). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrClientExtensions.cs | Adds GetService<T> + DataContent overload for OCR invocation. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrBoundingRegion.cs | Adds polygon-based geometry primitive + rectangle helper + bounds computation. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/OcrBlock.cs | Adds layout-block model (text/kind/geometry/confidence). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/IOcrClient.cs | Introduces OCR client abstraction (unary call + IProgress + GetService). |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Ocr/DelegatingOcrClient.cs | Adds delegating base class for OCR pipeline middleware. |
| src/Libraries/Microsoft.Extensions.AI.Abstractions/Microsoft.Extensions.AI.Abstractions.json | Updates API baseline to include new OCR abstraction surface. |
Comment on lines
+27
to
+31
| public OcrBoundingRegion(int pageNumber, IReadOnlyList<float> polygon) | ||
| { | ||
| PageNumber = pageNumber; | ||
| Polygon = Throw.IfNull(polygon); | ||
| } |
Comment on lines
+16
to
+32
| /// <summary>Registers a singleton <see cref="IOcrClient"/> in the <see cref="IServiceCollection"/>.</summary> | ||
| /// <param name="serviceCollection">The <see cref="IServiceCollection"/> to which the client should be added.</param> | ||
| /// <param name="innerClient">The inner <see cref="IOcrClient"/> that represents the underlying backend.</param> | ||
| /// <param name="lifetime">The service lifetime for the client. Defaults to <see cref="ServiceLifetime.Singleton"/>.</param> | ||
| /// <returns>An <see cref="OcrClientBuilder"/> that can be used to build a pipeline around the inner client.</returns> | ||
| public static OcrClientBuilder AddOcrClient( | ||
| this IServiceCollection serviceCollection, | ||
| IOcrClient innerClient, | ||
| ServiceLifetime lifetime = ServiceLifetime.Singleton) | ||
| => AddOcrClient(serviceCollection, _ => innerClient, lifetime); | ||
|
|
||
| /// <summary>Registers a singleton <see cref="IOcrClient"/> in the <see cref="IServiceCollection"/>.</summary> | ||
| /// <param name="serviceCollection">The <see cref="IServiceCollection"/> to which the client should be added.</param> | ||
| /// <param name="innerClientFactory">A callback that produces the inner <see cref="IOcrClient"/> that represents the underlying backend.</param> | ||
| /// <param name="lifetime">The service lifetime for the client. Defaults to <see cref="ServiceLifetime.Singleton"/>.</param> | ||
| /// <returns>An <see cref="OcrClientBuilder"/> that can be used to build a pipeline around the inner client.</returns> | ||
| public static OcrClientBuilder AddOcrClient( |
Comment on lines
+45
to
+63
| /// <summary>Registers a keyed singleton <see cref="IOcrClient"/> in the <see cref="IServiceCollection"/>.</summary> | ||
| /// <param name="serviceCollection">The <see cref="IServiceCollection"/> to which the client should be added.</param> | ||
| /// <param name="serviceKey">The key with which to associate the client.</param> | ||
| /// <param name="innerClient">The inner <see cref="IOcrClient"/> that represents the underlying backend.</param> | ||
| /// <param name="lifetime">The service lifetime for the client. Defaults to <see cref="ServiceLifetime.Singleton"/>.</param> | ||
| /// <returns>An <see cref="OcrClientBuilder"/> that can be used to build a pipeline around the inner client.</returns> | ||
| public static OcrClientBuilder AddKeyedOcrClient( | ||
| this IServiceCollection serviceCollection, | ||
| object? serviceKey, | ||
| IOcrClient innerClient, | ||
| ServiceLifetime lifetime = ServiceLifetime.Singleton) | ||
| => AddKeyedOcrClient(serviceCollection, serviceKey, _ => innerClient, lifetime); | ||
|
|
||
| /// <summary>Registers a keyed singleton <see cref="IOcrClient"/> in the <see cref="IServiceCollection"/>.</summary> | ||
| /// <param name="serviceCollection">The <see cref="IServiceCollection"/> to which the client should be added.</param> | ||
| /// <param name="serviceKey">The key with which to associate the client.</param> | ||
| /// <param name="innerClientFactory">A callback that produces the inner <see cref="IOcrClient"/> that represents the underlying backend.</param> | ||
| /// <param name="lifetime">The service lifetime for the client. Defaults to <see cref="ServiceLifetime.Singleton"/>.</param> | ||
| /// <returns>An <see cref="OcrClientBuilder"/> that can be used to build a pipeline around the inner client.</returns> |
Adds an optional OcrImage sink so engines can return page images/figures when OcrOptions.IncludeImages is requested. Every member is optional: document-native engines (Mistral OCR inline images, Azure Document Intelligence figures) populate Content with rendered bytes; a vision-LLM transcriber may populate only Caption. One shape serves both provider archetypes.
Symmetric with the existing DataContent overload. Handles self-contained data: URIs by delegating to the DataContent path and does no file/network IO. For file: and remote URIs it throws NotSupportedException, leaving native URL passthrough (Mistral document_url, Azure DI uriSource) as an explicit open design question.
The method returns a rich OcrResult (markdown, tables, figures/images, blocks, confidence, language), not plain text, so GetTextAsync was a misnomer. ExtractAsync names the document-extraction operation accurately. Scoped to the Ocr/ types only; ISpeechToTextClient.GetTextAsync is unrelated and unchanged. Alternatives considered for maintainer discussion: GetDocumentAsync, AnalyzeAsync.
…ines Complete the source rename across the OCR test suites in both Microsoft.Extensions.AI.Abstractions.Tests and Microsoft.Extensions.AI.Tests, and regenerate the two API baselines (which also reflect the round-2 members ExtractFromUriAsync, OcrImage, and OcrPage.Images). Also fix pre-existing StyleCop/Sonar violations in the OCR tests that were blocking the core test project from compiling.
Stream.CopyToAsync(Stream, CancellationToken) does not exist on net462/ netstandard2.0, so the 2-arg call failed to compile (CS1503) on the net462 leg. Use the #if !NET conditional buffer-size overload, matching the existing DataContent.cs convention.
Introduce OcrPoint and OcrBoundingBox readonly record structs and retype OcrBoundingRegion.Polygon as IReadOnlyList<OcrPoint> with a typed OcrBoundingBox GetBounds() return. This makes odd or empty vertex lists unrepresentable (fixing the GetBounds inverted-bounds edge case) and reads self-documenting, while still carrying a rotation-skewed quadrilateral. Rename OcrPage.Index (zero-based) to OcrPage.PageNumber (one-based) so it correlates directly with OcrBoundingRegion.PageNumber, matching Azure Document Intelligence, PdfPig, and human page conventions, and removing the off-by-one adjustments providers had to make. Update the internal OcrDocumentReader and the OCR tests, and regenerate the Microsoft.Extensions.AI.Abstractions API baseline.
Family-consistency changes surfaced by an API-review rehearsal against the sibling abstractions (IChatClient, ISpeechToTextClient): - Streaming: add IAsyncEnumerable<OcrResponseUpdate> via ExtractStreamingAsync (the unary/streaming/GetService triad, matching IChatClient) plus a ToOcrResult/ToOcrResultAsync reducer; retire IProgress<OcrProgress> and the OcrProgress type (progress folds onto OcrResponseUpdate). - Unseal the OCR DTOs (OcrOptions, OcrResult, OcrPage, OcrBlock, OcrTable, OcrTableCell, OcrImage, OcrBoundingRegion, OcrUsage) to mirror the unsealed ChatResponse/ChatOptions family shape. - Geometry units: add OcrPage.Width/Height and an OcrCoordinateUnit (ChatRole-style open struct: Pixel/Inch/Normalized) so bounding coordinates are interpretable across engines. - Model OcrBlock.Kind and OcrTableCell.Kind as ChatRole-style open structs (OcrBlockKind, OcrTableCellKind) instead of raw strings. - OcrBoundingRegion.GetBounds() now returns OcrBoundingBox? (null on empty polygon) instead of an ambiguous all-zero default. - Remove the leaky OcrResult.OcrSource; document OcrTable.Cells as authoritative when non-null. - Add <exception> docs to the AddOcrClient/AddKeyedOcrClient overloads. Updates the middleware (Logging/OpenTelemetry/ConfigureOptions), the MEDI OcrDocumentReader consumer, OCR tests, and both API baselines accordingly.
…es, usage tokens Continue the IOcrClient reshape ahead of API review (dotnet#7587/dotnet#7588). - Replace parallel OcrPage.Blocks/Tables/Images with a single reading-order Elements list over a new polymorphic OcrElement base (OcrBlock/OcrTable/OcrImage derive); consumers project with OfType<T>(). - Add optional nested OcrTableCell.Elements for structured cell content; keep the Content string as a flat-text convenience. - Convert OcrCoordinateUnit to a closed enum and add Point; add an OcrCoordinateOrigin enum; move both to document level (OcrResult/OcrPageResult) and remove the per-page CoordinateUnit. - Keep OcrBlockKind and OcrTableCellKind as open structs; add RowHeader and RowSection well-known cell kinds. - Add nullable Input/Output/TotalTokenCount to OcrUsage for the vision-LLM path. - Rename Markdown to Text on OcrPage/OcrResult; drop response ModelId, OcrPage.Confidence, and OcrOptions.IncludeImages; FromRectangle takes float. - Keep ToOcrResult/ToOcrResultAsync reducers and the convenience overloads as extension methods, matching the MEAI ChatResponseExtensions precedent. - Regenerate the ApiChief baselines for the reshaped surface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
SPIKE-07/08 primary-source verification refuted the round-1 assumption that a
document uses one uniform coordinate unit. Google Document AI models
Page.Dimension{width,height,unit} per page, and Azure DI's per-page
DocumentPage.Unit is "pixel" for image inputs and "inch" for PDF, so one
analyze call over a mixed batch returns different units on different pages.
- Group OcrPage.Width/Height into a new OcrPageDimensions readonly record
struct (the extent; an atomic pair several engines bundle).
- Move CoordinateUnit + CoordinateOrigin from the document level (OcrResult,
OcrPageResult) onto OcrPage; they are siblings to the dimensions extent,
together describing the per-page coordinate system.
- Drop the reducer's last-non-null coordinate carry-forward: pages already
carry their own unit/origin.
- Defer continuous page-rotation angle (Azure-family-only) to
AdditionalProperties for v1; element rotation is already carried losslessly
by the polygon OcrBoundingRegion.
- Update per-page tests and regenerate the ApiChief baseline (32 public types).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
SPIKE-06 built an exhaustive 14-engine raw-output inventory to justify the RawRepresentation/AdditionalProperties escape hatches, then a promote interview resolved which recurring un-homed fields become first-class. - Add BoundingRegion, Confidence, RawRepresentation, and AdditionalProperties to OcrTableCell, mirrored member-for-member from OcrElement. Five engines (Textract, Google Document AI, Azure DI, Adobe, Docling) emit per-cell geometry, and a cell has its own rectangle even when empty or padded. OcrTableCell deliberately does not derive from OcrElement (a cell is not a page-reading-order element) and no shared base type is extracted yet; mirroring the members keeps that a reversible way-station under [Experimental]. - Add [JsonIgnore] object? RawRepresentation to OcrPage. Every other node carries it; the page node was the lone exception. Because the page rides through ToOcrResult reduction, provider-native page data now survives into OcrResult.Pages, closing a silent per-page raw data-loss. - Detected language is deferred (not promoted) to a standardized AdditionalProperties key rather than a lossy typed field. - Add OCR tests for cell geometry round-trip and per-page raw survival; regenerate the ApiChief baseline (five additive members, no new types). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
Move the reshaped OCR surface out of Microsoft.Extensions.AI(.Abstractions) into a standalone peer library, Microsoft.Extensions.DocumentExtraction, mirroring VectorData / DataIngestion. OCR is document extraction rather than an AI-branded concern, so it gets its own package + namespace (Microsoft.Extensions.DocumentExtraction) with domain-noun naming (Document*/DocumentExtraction*) instead of the MEAI-internal verb scheme. - New Microsoft.Extensions.DocumentExtraction.Abstractions: the Document* model, IDocumentExtractionClient, DocumentExtractionResult/Options/ ClientMetadata/PageResult, extensions, DelegatingDocumentExtractionClient. - New Microsoft.Extensions.DocumentExtraction: builder + Logging/ OpenTelemetry/ConfigureOptions delegating clients + DI/builder extensions, with self-contained internal telemetry helpers (no MEAI-core coupling). - Remove the OCR types from Microsoft.Extensions.AI(.Abstractions) and shrink both API baselines accordingly (only OCR type-blocks removed; the manual params edits and JSONC comments are preserved byte-for-byte). - Populate the two new Stage=normal baselines (peer precedent: VectorData ships a populated baseline for its [Experimental] surface). - Repoint Microsoft.Extensions.DataIngestion's OcrDocumentReader onto DocumentExtraction.Abstractions. - Add DiagnosticIds MEDE0001 for the new [Experimental] surface. - Retarget all OCR tests to two new DocumentExtraction test projects. Green across all five TFMs (netstandard2.0, net462, net8.0, net9.0, net10.0); DocumentExtraction tests 31/31 + 29/29 on net8/9/10; the MEDI OcrDocumentReader test stays green after the repoint. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
The OCR test helper moved to Microsoft.Extensions.DocumentExtraction.Abstractions.Tests (TestDocumentExtractionClient.cs) during the extraction, but Microsoft.Extensions.AI.Tests.csproj still linked the moved-away ..\Microsoft.Extensions.AI.Abstractions.Tests\TestOcrClient.cs, causing error CS2001 (source file not found) on all TFMs in CI. No MEAI test references any Ocr type, so the link is safe to delete. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
Every packable library in the repo ships a README.md (auto-included as PackageReadmeFile on pack); the two new DocumentExtraction packages were missing theirs, so packing failed with NU5019. Add READMEs mirroring the DataIngestion peers, describing the document-extraction capability and the Abstractions-vs-core package split. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 19ea13dd-f444-4942-b751-162711478580
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces
Microsoft.Extensions.DocumentExtraction, a provider-neutral document-extraction capability (OCR, layout, tables, figures, native-PDF text) as its own peer library — a sibling toMicrosoft.Extensions.VectorDataandMicrosoft.Extensions.DataIngestion, rather than a capability added toMicrosoft.Extensions.AI. It follows the same abstraction + builder + middleware + DI shape as the M.E.AI capability family (IChatClient,ISpeechToTextClient, etc.) and may referenceMicrosoft.Extensions.AI.Abstractionsinternally (e.g.DataContentinputs) without carrying the "AI" brand in its own namespace or types.Implements the API proposal in #7587.
Why its own library (2026-07-30)
Per review steer, document extraction reads as its own domain rather than an
IChatClientsibling: it pulls structured content out of documents, complementingMicrosoft.Extensions.DataIngestion, which feeds content into RAG. Modeling it as a peer library — likeVectorDataandDataIngestion, both of which reference M.E.AI internally without being part of it — keeps the capability, its future provider packages, and its ownership independent of the AI abstractions, and lets a provider team (for example, Azure AI Document Intelligence) implementIDocumentExtractionClientwithout taking an "AI" branding dependency.Naming flips accordingly, from the M.E.AI-internal verb scheme (
SpeechToText) to the domain-noun convention of the peers (VectorStore,IngestionDocument): a neutralDocument*content model plusDocumentExtraction*operation/client types. The neutralDocument*model is kept extraction-agnostic so a later hoist into a shared home co-owned with DataIngestion stays a namespace-forward rather than a redesign (cross-team track; does not block this PR).What's included
Abstractions (
Microsoft.Extensions.DocumentExtraction.Abstractions)IDocumentExtractionClient,DelegatingDocumentExtractionClient,DocumentExtractionClientExtensions,DocumentExtractionClientMetadataDocumentExtractionOptions,DocumentExtractionResult,DocumentPage,DocumentExtractionPageResult(the per-page streaming unit)DocumentElement(polymorphic base) withDocumentBlock,DocumentTable,DocumentImagederiving;DocumentTableCellDocumentBoundingRegion,DocumentBoundingBox,DocumentPoint,DocumentPageDimensionsDocumentBlockKind,DocumentTableCellKind(open structs);DocumentCoordinateUnit,DocumentCoordinateOrigin(enums)DocumentExtractionUsage,DocumentExtractionPageResultExtensions(streaming reducers)Middleware + DI (
Microsoft.Extensions.DocumentExtraction)DocumentExtractionClientBuilder,AsBuilder,AddDocumentExtractionClient/AddKeyedDocumentExtractionClientLoggingDocumentExtractionClient,OpenTelemetryDocumentExtractionClient,ConfigureOptionsDocumentExtractionClient+UseLogging/UseOpenTelemetry/ConfigureOptionsbuilder extensions (mirrors theISpeechToTextClienttemplate)Both packages multi-target down to netstandard2.0 (peer precedent:
DataIngestion/VectorData). All public surface is[Experimental]under the newMEDE0001diagnostic id (peer precedent: VectorData =MEVD9001). Unit tests cover both libraries.Reshape since the initial proposal (2026-07-22 API review)
Following the API review plus a 12-engine provider survey, the result model was reshaped before the library move. Full rationale is in the updated #7587; the highlights (names below are the post-move
Document*/DocumentExtraction*names):DocumentPage.Blocks/Tables/Imagescollapse into one reading-orderDocumentPage.Elementslist over a polymorphicDocumentElementbase; consumers project withOfType<T>(). Optional nestedDocumentTableCell.Elementscarries structured cell content.DocumentElementis shaped to be promotable to a future shared document-element type (cross-team track; does not block this PR).DocumentCoordinateUnitis a closedenum { Pixel, Point, Inch, Normalized }and a newDocumentCoordinateOriginenum { TopLeft, BottomLeft }is added; both live per page onDocumentPage— engines emit different units for different pages (e.g. a batch mixing image and PDF inputs), per GooglePage.Dimensionand Azure DIDocumentPage.Unit.DocumentPage.Width/Heightgroup into aDocumentPageDimensionsvalue type (the extent), a sibling to unit/origin (which together describe the coordinate system). A continuous page-rotationangleis deferred toAdditionalPropertiesfor v1 (element rotation is already carried by the polygonDocumentBoundingRegion).DocumentBlockKind,DocumentTableCellKind) — the survey showed these grow per engine;RowHeader/RowSectionadded to the cell kinds.DocumentExtractionUsagecarries nullableInputTokenCount/OutputTokenCount/TotalTokenCountfor the vision-LLM path.OcrResponseUpdate→DocumentExtractionPageResult;ExtractStreamingAsync→ExtractPagesAsync;Markdown→Text; the streaming update'sPageis now non-null (no sentinel) andStatusis dropped; response-sideModelId, pageConfidence, and optionsIncludeImagesare removed;DocumentBoundingRegion.FromRectangletakesfloat.RawRepresentation/AdditionalProperties; from it,DocumentTableCellgainsBoundingRegion/Confidence/RawRepresentation/AdditionalProperties(mirrored fromDocumentElement— 5 engines emit per-cell geometry) andDocumentPagegainsRawRepresentation(symmetry; survives streaming reduction). Detected language is deferred to a standardizeddetectedLanguagesAdditionalPropertieskey rather than a lossy typed field.Notes
Microsoft.Extensions.Http.Resilience); the.Use(...)primitive can still wrap a custom decorator.DocumentBoundingRegion).System.Numerics.Vector2has no region type;System.Drawing.RectangleFis axis-aligned only (it can't hold a skewed quad) and the BCL has no polygon primitive; reusingSystem.Drawing.PointFforDocumentPointalone would delete no type while coupling coordinates to a drawing namespace.GetServicestays onIDocumentExtractionClient(SPIKE-04/05). A single optional accessor covers three independent needs: the provider metadata the shipping OTel/logging middleware reads to tag spans, the provider-native SDK escape hatch (e.g. Azure Document Intelligence), and unwrapping a hiddenIChatClient.AsDocumentExtractionClient()vision adapter back to its inner chat client. Dropping it would force a mandatoryMetadataproperty and still leave the escape hatch/unwrap unsolved; kept for v1 (reversible while[Experimental]).Deferred follow-ups (explicit non-goals this PR)
AIJsonUtilities.DefaultOptionsreflection fallback for now (matchingVectorData/DataIngestion, which ship no source-gen context); a package-owned source-gen context for full AOT parity is a follow-up.Document*model into a shared home co-owned withDataIngestion; this PR keepsDocument*neutral so that stays a namespace-forward.IDocumentExtractionClientbelongs in a provider-owned package (mirroringDataIngestion.MarkItDown/.Markdig), not here.[Experimental]name churn. Full 5-TFM public-API baselines are regenerated once the library name/placement is confirmed; any review-driven rename is a mechanical re-rename + baseline regen, not a redesign.Microsoft Reviewers: Open in CodeFlow